The game of poker presents a serious challenge to artificial intelligence research. Uncertainty in the game stems from partial information, unknown opponents, and game dynamics dictated by a shuffled deck. Add to this the large space of possible game situations in real poker games such as Texas Hold'em, and the problem becomes very difficult indeed. Human players, and even the best computer players, are certainly not optimal in that idiosyncratic weaknesses associated with the human or computer players can be exploited to obtain higher pay offs as compared to an approximating approach utilizing linear programming techniques.
Existing approaches to opponent modeling have employed a variety of approaches including reinforcement learning, neural nets and frequent statistics. Additionally, earlier work on using Bayesian models for poker has attempted to classify the opponent's hand into one of a variety of broad hand classes. They did not model uncertainty in the opponent's strategy, utilizing instead an explicit strategy representation. This strategy was updated based on empirical frequencies at play, but these models showed little improvement due to this updating. Other systems utilized Bayesian probabilistic models for Hold'em poker games wherein the uncertainty in the game and the opponent was completely modeled.
In certain circumstances and in certain games, it is desirable to decrease the element of skill element and increase the chance element. For example, when a game such as poker is implemented as a house-banked game, such as in a gaming machine, it may be desirable that the game be a game where chance, rather than skill, is a factor in determining the outcome. That is, where a game is implemented in a gaming machine, the player competes against the gaming machine. If the gaming machine adapts to the player's skill, then this arguably becomes a game of skill and may not be desirable for two reasons. First, the gaming machine would arguably play “too well” and only expert players or players who are exceptionally fortunate would have a chance to win against the gaming machine. Second, under some states' gaming regulations, games of skill are only allowed when the game is “player-banked” and only non-casino, human players are competing against one another. Thus, casino poker is typically played in poker rooms in which multiple players each contribute to a pot. The pot, minus a “rake” retained by the house for hosting the game, is awarded to the winning player.
Among the problems in modeling poker play are the large number of unknowns, such as the cards that have not been dealt at any point in the game, and the multiple options available to a player at various points in the game. Another unknown is the role of “bluffing” by a player.
For example, in a typical hand of Texas Hold'em poker, a single game consists of a number of stages separated by decision nodes where a decision must be made by a player. At a first stage, the pot is seeded. In one embodiment, a blind bet is placed by at least one player. In alternate embodiments, multiple blind bets, e.g. a small blind and a large blind, are received from different players. Optionally, the blind bet(s) rotates among players. In an alternate or additional optional embodiment, each participating player may place an ante wager. The blind bet(s) and/or ante wagers are aggregated to a pot and each player is dealt a hand of cards.
In conventional Texas Hold'em, the hand consists of two cards. This is typically followed by a round of betting. In a typical game of Texas Hold'em, the actions available to a player are to: (a) fold, e.g. terminate play by the player; (b) bet or check, if no other player has previously placed an additional wager, or call, if another player has previously placed an additional wager; or (c) raise, if another player has placed an additional wager and the player wishes to increase the amount of the additional wager. Bets, calls, and raises are aggregated to the pot. In some versions, the number of raises, as well as the size of the bets and raises, is limited.
Additional stages are conducted in which community cards available to all the players on constructing the player's final hand are revealed. In conventional Texas Hold'em, a total of five community cards are revealed in three stages, with each stage followed by a round of betting. As known in the art, three cards (referred to as the “flop”) are revealed in a stage, one card (referred to as the “turn”) is revealed in an ensuing stage, and one card (referred to as the “river”) is revealed in a later ensuing stage. If players fold thereby leaving only one active player, the active player wins. If more than one player remains in the game through the revelation of all the community cards and the rounds of betting, each player forms a final hand using five of the seven cards (two in the player's hand plus five community cards) available to the player. The final hands are compared, and at least a portion of the pot is awarded to the player with the highest ranking poker hand.